Manage 
your market 


data with 
TIMESQUARE. 


Introducing TimeSouare, the truly open time series data manager. 
It works with any source. And connects to any environment. 





It’s a first in the marketplace. TimeSQuare brings together acquisition, manage- 
ment, access and distribution of time series data in a single, open systems 
environment. TIMESQUARE is designed for managing financial market data and is 
optimized for efficient update and fast access of time-based information. 
æ Integrated middleware provides query and data distribution to or from any 
environment. TIMESQUARE lets you: 


e Acquire and process market data from any source—external or internal 


e Create and manage large amounts of time series and related data in a UNIX 
environment 


e Distribute data to any platform like DB2 on the mainframe, Sybase in UNIX 
or Oracle in Windows NT 


* Query, research and summarize data easily using popular PC packages like — 
Excel or Lotus 


TIMESQUARE™ 





TIMESQUARE 


SOL for Market Data Access 
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Market Overview 


Standard Query Language (SQL) is the lingua franca of 
data access. Relational databases and data warehouses pri- 
marily use SQL for data management and retrieval. How- 
ever, the financial marketplace continues to be challenged 

y the dimension of time with regards to data storage and 
access. SQL does not support the concepts of analysis over 
time, ranking or trading day calendars. The combination of 
these deficiencies with the inefficiencies of the relational 
model have necessitated a new solution for managing 
large amounts of complex, time-based data. 


TimeSquare, an open systems manager for financial mar- 
ket data, addresses these issues. 
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Introduction 


Conventional relational databases are inefficient for the 
storage of time series data. The two-dimensional column 
and row structure of relational databases forces the repre- 
sentation and storage of time series data as repeated key 
and date values — something for which generic compres- 
@sion algorithms are not optimized. 


A conventional relational database management system 
cannot support efficient access of time series data for both 
point-in-time queries and over-time queries. When access 
is optimized along one dimension, the reverse dimension 
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exhibits worst-case performance. For example, a relational 
database optimized for efficient access of a few quotes over 
a long time frame is extremely slow at retrieving today's 
quotes for an entire exchange. A typical solution involves 
strategies such as temporal partitioning, where one set of 
tables is set up for current market quotes and another is set 
up for history. 


Financial institutions, having adopted the relational 
model, have been forced to accept excessive storage 
requirements, complex data management infrastructures 
and slow database updates in order to support large cen- 
tralized collections of time series data. 


SQL was initially developed to support relational data- 
bases. It remains the industry standard for accessing data. 
However, time series concepts such as time consolidation, 
ranking and moving averages require either extremely 
complex and inefficient SQL queries or significant post- 
processing by the client application submitting the query. 


As a result, many financial institutions have developed 
their own systems at great cost, or acquired commercial 
proprietary systems that lack the openness and SQL sup- 
port of standard relational database products. The result- 
ing operational systems are often limited in ways that 
relational database systems are not. Developers and end- 
users are required to learn proprietary fourth generation 
languages. In many cases, a standard relational database 
system is required in conjunction with the proprietary time 
series solution. 


TimeSquare is the first solution focused on centralizing 
large collections of market data, addressing the problems 
with the relational model, and extending SQL for time 
series data management. 
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TimeSquare 


TimeSquare is the first relational database product specifi- 
cally developed and optimized for managing large central- 
ized collections of market data. It addresses the 
shortcomings of conventional relational database manage- 
ment systems without compromising relational database 
and open system standards. 


TimeSquare features a high performance engine designed 
specifically to process large time series databases. It deliv- 
ers a set of proprietary extensions to SQL enabling sophis- 
ticated time series calculations at the server. These 
extensions are combined with a calendar system that sup- 
ports trading days in multiple markets. 


TimeSquare is the first off-the-shelf product that under- 
stands the flow of market data through the financial enter- 
prise. It includes: 


@ a data quality module 


e a scheduler for automating the processes of data 
updates and exports. 


e vendor-specific data loaders. 
e integrated middleware for export of TimeSquare data. 


e operational tools for system, database, and data admin- 
istration. 


es 
Extending the Relational Structure 


TimeSquare incorporates a market data optimized data- 
base engine that minimizes the redundancy inherent in the 
relational structure. It is tuned to handle the particular 
@iemands of market data update schedules and queries. 


TimeSquare optimizes both point-in-time and over-time 
access internally. Because partitioning is implemented as 
an integral component of the database engine, the process 
is transparent. A time series table is presented as a single 
relational table accessible by standard or time series 
extended SQL. Queries are analyzed and optimized at the 
engine. 


Optimized compression and integrated partitioning, make 
TimeSquare the first truly relational time series database 
manager—one that is able to efficiently store and support 
such diverse data types as time series, reference data, cor- 
porate actions and terms and conditions 





TimeSquare uses proprietary compression techniques to 
compress time series data. The result is faster updates, 
higher compression ratios, and reduced read/write over- 
head. 
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Extending SQL 


TimeSquare provides an extended SQL which understands 
the notion of time series. TimeSquare SQL (TSQL) repre- 
sents the combination of standard SQL with extensions to 
address the unique needs of time series and financial mar- 
ketplace concepts. Financial time series require these exten- 
sions because, unlike two-dimensional data typically 
stored in relational databases, a three-dimensional struc- 
ture exists. Typically this structure is referenced by instru- 
ment (for example, ticker), facts (for example, open, high, 
low, and close) and period (for example, daily data for five 
years). 


Given this three-dimensional structure, TimeSquare intro- 
duces through TSQL the concept of time series relational 
tables. These tables are specifically structured and opti- 
mized for the storage, retrieval and update of time series 
data. TSQL includes extensions for creating and manipu- 
lating these tables. 


Additionally, it provides time series extensions to SQL that 
support: 


e “trading day knowledgeable" aggregations, such as, 
MIN, MAX, SUM, COUNT, FIRST and LAST. For exam- 
ple, find the maximum value for each month. 


e logical extensions for comparison over time. For exam- 
ple, display all values greater than 10% above a previous 
period. 


¢ variable time windows for such operations as moving 
averages and cumulations. For example, calculate 5-day 
moving averages or two week cumulative volumes. 


e forward filling. For example, fill missing values with the 
value from the previous trading day. 


e ranking. For example, display the top 10 movers. 


e multiple calendars and calendar operations such as 
union, intersection and difference. For example, display 
data for common trading days in London and New 
York. 
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The Time Series Table 


Syntax: 


CREATE TABLE table_name 


A TimeSquare database is comprised of two kinds of rela- 
tional tables: static tables and time series tables. 


Static tables are standard relational tables found in any 
relational database. They store any data that can be 
uniquely referenced by its position in a column and row 
structure. They are defined by using the standard SQL 


TIMESERIES 

RETAIN N 

(column_name1 data type (length), 
column_name2 data type (length), 

INSTRUMENT (column_nameX,column_namey....), 
PERIOD column_name) 


CREATE TABLE statement. 
Syntax: 


CREATE TABLE table_name 
(column! data type (length), 
column2 data type (length), 

. ep o»o»sopgo oo’ ) 


Example: 


REATE TABLE ISSUES 
(TICKER CHAR (6), 
EXCHANGE CHAR (6), 
CUSIP CHAR(9) ) 


Time series tables are optimized tables that store regular 
events over time. They are partitioned into current and his- 
tory segments to provide optimal access for either: 


e Point-in-time queries (many quotes over a short time 
frame) 


e Over-time queries (few quotes over a long time frame) 


To support this structure, the CREATE TABLE statement has 
been extended to include the keywords: 


TIMESERIES 
r o indicate a time series table. 


RETAIN N 
To indicate the minimum number of days that records 
should be kept in the current partition. 


PERIOD column_name 
To identify the column in the table that holds the time 
dimension of the instruments. 


INSTRUMENT (columnX,columny,...) 
To specify the column or combination of columns that 
uniquely identify each instrument in the table. 


The following syntax is used to create a time series rela- 
tional table. 


Example: 


CREATE TABLE ISSUES 
TIMESERIES 
RETAIN 7 
(TICKER CHAR(6), 
EXCHANGE CHAR (6), 
CUSIP CHAR(9), 
TDATE DATE, 

CLOSE DOUBLE 

OPEN DOUBLE 
INSTRUMENT (CUSIP), 
PERIOD TDATE) 
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Calendaring 


Calendaring is another unique aspect of the financial mar- 
ketplace. Not usually found in standard relational data- 
bases, calendaring is critical for market data time series. 
Calendaring is used to keep track of market trading days 
making calculations, data manipulation and data screen- 
ing much easier. 


TimeSquare contains a calendar table which stores the col- 
lection of calendars. TimeSquare supplies a standard set of 
calendars for the primary global markets. Calendars are 
easily defined and inserted into the table using TSQL. 


Calendars do not have to be stored. They can be defined as 
expressions and used in SELECT statements. TSQL pro- 
vides an advanced calendaring expression language with 
functions for: 


e Delimited calendars - specifying a start and end date fora 
calendar using SINCE and UNTIL. 


e Periodic Calendars - defining a trading day pattern for a 
calendar using DAY OF, WEEK OF, MONTH OF, and 
QUARTER OF. 


è Date calendars - defining individual dates that affect the 
trading day pattern including holidays and other non- 
trading days using EACHYEAR and DATES. 
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e Combined Calendars - combining calendars for joining, 
intersecting and differentiating trading patterns 
between markets using AND, OR, NOT, BEFORE, AFTER 
or INTO. 


Delimited Calendars 
TimeSquare assumes a total date range from January 1, 
1850 to December 31, 2100. The functions SINCE and 


UNTIL used either separately or together, select data either 
since a specified date or until a specified date. For example: 


(SINCE ‘1994-1-1') AND (UNTIL ‘1996-7-31') 


Periodic Calendars 


The following periodic calendar functions are used to spec- 


oO trading day patterns. 

Syntax: 

(n1,n2,...) DAY OF period 

(n1,n2,...) WEEK OF period 

(n1,n2,...) MONTH OF period 

(n1,n2,...) QUARTER OF YEAR 

For these functions, n equals any positive or negative num- 
ber or set of numbers and period can be any valid period 
greater than the period function being used. Using these 


functions, date patterns can be set for accessing data. 


Examples 


(1,2,3,4,5) DAY OF WEEK | Monday to Frida 
Bahdiyd nd Sun 
(-1 DAY OF QUARTER) Last day of each quarter 


(1 WEEK OF QUARTER) First week (7days) of each 
quarter 
(1,2) WEEK OF MONTH) First two weeks (14 days) of 
each month 
ear. 


(-2 MONTH OF YEAR) November of each year 













(3 QUARTER OF YEAR) Third quarter of each year 


Date Calendar Functions 


Functions are included to define calendar expressions of 
individual dates. These expressions may then be used in 
defining holidays and unplanned non-trading days for 
specific markets. 


The EACHYEAR function returns a collection of annual 
dates. For example: 


EACHYEAR ‘12-25’ 


This expression returns every December 25 and can be 
saved as the Christmas holiday calendar. 


The DATES function is similar to EACHYEAR, except it is 
used to define calendar expressions of individual dates. For 
example: 


DATES ‘1996-06-19’ 


This returns the single date, June 19, 1996. 


Combining Calendar Functions 


The power of the TSQL calendar functions is in its ability to 
combine expressions to form complete trading calendars 
for a given market or exchange. With the functions 
BEFORE, AFTER and INTO almost any set of dates can be 
defined. 


Calendar expressions are defined and inserted into the 
TimeSquare calendar table called CAL . DEF. For example: 


INSERT INTO CAL_DEF 
(NAME, EXPRESSION, TYPE, REMARK) 
VALUES (‘CAL_USBD’, 
“(1,2,3,4,5) DAY OF WEEK’, 
‘TRADE’, ’BASIC U.S. TRADING DAYS’ ) 


The above statement inserts the calendar. expression, 
(1,2,3,4,5) DAY OF WEEK into the TimeSquare calen- 
dar table as a TRADE date under the calendar name 
CAL_USBD. The value of TYPE must be one of TRADE, 
HOLIDAY or NOTRADE. The TYPE attribute defines 
whether the expression is inclusive (TRADE) or exclusive 
(NOTRADE and HOLIDAY). An unlimited number of expres- 
sions can be saved under the same name, CAL_USBD, to 
form the complete U.S. trading calendar. 
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The BEFORE and AFTER functions are used to create a cal- 
endar that consists of days in one calendar that fall on or 
after the dates in another calendar. 


For example: 


CAL.US_TRADING_DAYS is the calendar of U.S. trading 
days. 


The expression: 


CAL.US_TRADING_DAYS AFTER (1 DAY OF MONTH) 
would be a calendar of the first available U.S. trading day 
on or after the first day of every month. 


Alternatively: 


CAL.US_TRADING_DAYS BEFORE (-1 DAY OF MONTH) 
would be a calendar of the first available U.S. trading day 
on or before the last day of every month. 


out example: 


((1,2,3,4,5) DAY OF WEEK) 

AFTER (EACHYEAR ‘12-25') 

returns the date of the Monday to Friday that falls on or 
after December 25. If the 25th is on DAY 1,2,3,4 or 5, 
then December 25 is returned. If it falls on DAY 6 or 7, 
then the following Monday is returned. 


ABLE 1: U.S. Exchange Trading Calendar 


aan cc 
CAL_USBD (1,2,3,4,5) DAY OF WEEK 


The INTO function expands on BEFORE and AFTER by 
allowing an argument for some nth period before or after 
a calendar. 


Syntax: 

n calendar! INTO calendar2 

The following example uses INTO to define the U.S. 
Thanksgiving as the fourth Thursday on or after Novem- 
ber 1 of each year. 

Example: 


4(4 DAY OF WEEK) INTO EACHYEAR ‘11-1’ 


Table 1, at the bottom of the page, is an example of a U. S. 
exchange trading calendar. 


ee 
Time Series Access 


The standard SQL SELECT statement is used to access 
and manipulate data. There are six primary clauses used 
with SELECT. 


TRADING DAYS 


CAL_USBD (1,2,3,4,5) DAY OF WEEK AFTER EACHYEAR ‘1-1’ HOLIDAY NEW YEAR’S 
CAL_USBD 3(1 DAY OF WEEK) INTO EACHYEAR ‘2-1’ HOLIDAY PRESIDENT’S DAY 


CAL_USBD 
CAL_USBD 


EACHYEAR ‘7-4’ 


can facie 


CAL_USBD (1 DAY OF WEEK) AFTER EACHYEAR ‘9-1’ HOLIDAY LABOR DAY 


4(4 DAY OF WEEK) 
(EACHYEAR ‘12-25’ 


CAL_USBD 


AND (6 DAY OF WEEK) ) )} 


CAL_USBD 
OR((5 DAY OF WEEK) 
BEFORE (EACHYEAR ‘12-25’ 
OR ((1 DAY OF WEEK) 
AFTER (EACHYEAR ‘12-25’ AND (7 DAY OF WEEK) )) 


INTO EACHYEAR 


( 1 DAY OF WEEK) BEFORE ((-1 DAY OF MONTH) AND HOLIDAY | MEMORIAL DAY 
(5 MONTH OF YEAR) 


((1,2,3,4,5) DAY OF WEEK) AFTER 


HOLIDAY INDEPENDENCE DAY 
NOTRADE EXCHANGE CLOSED 
DUE TO FLOOD 


111-1’ 


HOLIDAY THANKSGIVING 


HOLIDAY CHRISTMAS. IF 
FALLS ON SATURDAY, 
THEN RETURN PREVI- 
OUS FRIDAY. IF 
FALLS ON SUNDAY, 
THEN RETURN FOL- 
LOWING MONDAY. 
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Syntax: 


SELECT select items 
FROM table 
WHERE search condition 
GROUP BY grouping columns 
HAVING search condition 
ORDER BY sort columns 


TSQL provides an additional clause, ARRANGE BY, that 
facilitates the processing of time series. ARRANGE BY is 
used to: 


e Ensure that values used in time series calculations are 
consecutively dated on the PERIOD column. 


e Ensure that the data used in the time series calculations 
are within instrument boundaries. 


e Ensure that the selected time period being queried is 
inclusive of the dates in the selected calendar(s). 


@.. ARRANGE BY clause is always processed immediately 
after the WHERE clause. It uses the following syntax. 


Syntax: 


ARRANGE BY period column 
[INCLUDING calendar] 
WITHIN instrument 


Example: 


SELECT TICKER, EXCHANGE, TDATE, CLOSE 
FROM TSPRICES 
WHERE TICKER = ‘IBM’ 
ARRANGE BY TDATE 
INCLUDING 
((1,2,3,4,5) DAY OF WEEK) 
AND (SINCE ‘1997-01-06’) 
o AND (UNTIL ‘1997-06-26’) 
WITHIN TICKER, EXCHANGE 





Ranking 


The relational structure has never been suitable for ranking 
data. For example, determining the top 10 gainers or losers 
for a given exchange is almost impossible to express, calcu- 
late and access on demand in standard SQL. Many IT 
departments and applications will pre-process and store 
these at given intervals. TSQL introduces the TOP clause. It 
merges the functionality of ORDER BY and GROUP BY. 





Syntax: 


TOP n ON sort column(s) [ASC | DESC] 
[BY grouping column(s)] [ASC |DESC] 


The TOP n ON part of the clause sorts query results in 
ascending or descending order based on values in the sort 
column(s) specified. It then returns the top n number of 
records in the query results. 


The optional BY grouping column(s) part of the clause first 
groups the results by the values in the grouping column(s), 
and then sorts the records based on the sort column(s), with 
the top n number of records for each group being returned. 


The following example finds the top 10 volumes by 
exchange for a given date. 


Examples: 


SELECT TICKER, EXCH, CLOSE, VOLUME 
FROM PRICES 
WHERE TDATE='1997-1-8' 
TOP 10 ON VOLUME 
BY EXCHANGE 


By combining TSQL calendaring with TOP, periodic aggre- 
gations are easily performed. Another example, returns the 
stock with the highest total volume by exchange, by week, 
for a one month period: 


SELECT TICKER, EXCHANGE, 
SUM (VOLUME), WEEK_ENDING 
FROM PRICES 
WHERE TDATE BETWEEN 
‘1997-1-1' AND ‘1997-1-31’ 
GROUP BY TICKER, EXCHANGE, 
(5 DAY OF WEEK)AFTER TDATE 
AS WEEK_ENDING 
TOP 1 ON VOLUME BY EXCHANGE, 
WEEK_ENDING 
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Time Series Functions 


TSQL expands the functionality of the aggregation func- 
tions MIN, MAX, AVG, SUM,and COUNT, in order to sup- 
port the concept of moving aggregations. In standard SQL, 
these functions return a single value for each period in a 
selected time frame. TSQL uses the exact same function 
names but distinguishes the standard aggregation function 
from the time series moving aggregation function by the 
context in which it is used. 
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Consider the standard SQL statement: 


SELECT TICKER, TDATE, AVG (CLOSE) 
FROM PRICES 
WHERE TDATE BEWTEEN ‘1997-01-01’ 
AND ‘1997-01-31’ 
GROUP BY TICKER, TDATE 


As standard relational databases do not implicitly recog- 
nize the concepts of time and trading days, the AVG calcu- 
lation above simply computes an average close using any 
data that may exist for the dates in the BETWEEN clause. 
For each ticker, the closing values for each of the dates 
where data exists between January 1 and January 31, 1997 
will be added and divided by the number of dates, return- 
ing a single average value for each ticker. There is no con- 
sideration of moving time frames, missing dates or null 
values from non-trading days, such as weekends. 


Using TSQL: 


ELECT TICKER, TDATE, AVG(3,CLOSE) 
FROM PRICES 

WHERE TDATE BETWEEN 
*1997-01-06’ AND ‘1997-01-31’ 
AND TDATE IN CAL.USBD 

ARRANGE BY TDATE 

INCLUDING CAL.USBD 

AND SINCE ‘1997-01-01' 

AND UNTIL ‘1997-01-31’ 

WITHIN TICKER 


For each ticker, the closing values for each of the dates 
where data exists between January 1 and January 31, 1997 
will be selected. The TDATE column is arranged in order. 
Missing dates since the 1st and until the 31st will be added. 
Only those trading days in the calendar USBD will be con- 
sidered and a three day moving average will be calculated 
for each valid trading day. 


( Y same moving functionality and syntax applies to MIN, 
X, SUM, and COUNT. 


Previous 


Financial markets are driven by price movements up and 
price movements down. Given this, it is necessary to easily 
formulate expressions with comparisons between periods. 
A unique feature of TSQL is the PREVIOUS function. PRE - 
VIOUS returns the value of the observation from some 
specified number of periods before a selected date. This 
value can then be defined as a column and used in formu- 
las and expressions. 





Syntax: 


PREVIOUS (n,expression) 
Example: 


SELECT TICKER, TDATE,CLOSE, 
PREVIOUS (1,CLOSE) AS PREV1_CLOSE, 
PREVIOUS (2, CLOSE) AS PREV2_CLOSE 
FROM PRICES 
WHERE TDATE BETWEEN 
1997-01-06’ AND ‘1997-01-13’ 
AND TICKER IN (’IBM’,‘HWP’) 
ARRANGE BY TDATE 
INCLUDING (1,2,3,4,5)DAY OF WEEK 
AND (SINCE ‘1997-01-06’ ) 

AND (UNTIL ‘1997-01-137) 
WITHIN TICKER 
ORDER BY TICKER 


The results of this query are: 


TABLE 2: Example of PREVIOUS function 
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Fill 


The FILL time series function is also unique to TSQL. It 
causes the most recent observation to be brought forward 
when no value exists for a specific period. It is useful for 
graphics and export to applications that do not support the 
concept of trading days and nulls. It uses the syntax: 


FILL (expression) 
For example, entering FILL(CLOSE) would replace any 


missing or null close values with the most recent value pre- 
vious to the missing or null date. 
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Summary 


TimeSquare fully understands and applies the dimension 
of time to database management without compromising 
relational database and open system standards. It achieves 
this through optimized extensions of the relational model 
and SQL. The financial marketplace can now leverage 
existing infrastructure and knowledge by centralizing 
market data in TimeSquare. 








Soliton Associates, an international company providing high performance solutions to customers worldwide. 


To find out more about TIMESQUAEE, please call us at the Soliton location nearest you. 


Soliton Associates Limited Brooklyn, New York Tel: (718) 783-9330 

® 44 Victoria Street, Suite 2100 Rochester, New York Tel: (716) 256-6466 
Toronto, Ontario, Canada The Netherlands Tel: +31-20-646-4475 
M5C 1Y2 via Internet to: sales@soliton.com 


Telephone: (416) 364-9355 
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TIMESQUARE: 


last and powerful, day in and day out, 


Its a server, integrated into your enterprise, that lets you consolidate your 
market data, manage it and use it over and over. Because of TiMESQUARES 
powerful, built-in middleware, you can send data to remote locations easily or 
query it from standard applications. TIMESQUARE gives you the speed and power 


to manage your data from start to finish... day in and day out. 


ACQUIRES DATA ----» MANAGES DATA ---------- > DISTRIBUTES DATA TO 
FROM ANY START TO FINISH ANY ENVIRONMENT 
SOURCE Creates and maintains databases Mainframe, UNIX or PC 

h A À A Stores and updates data IBM, Sun, HP DEC aS © 
gSeongs, Kooba l Schedules database events TCP/IP, AppleTalk, Netbios, SPX/IPX 
i i ce 1 Nae Browses and edits data DB2, Sybase, Oracle, Informix, Ingres 
H Foo : ay : ý “i Monitors data quality AKEE DATA 
i TANS, : : vi ` Reports event status EASY TO USE 
tt fy z ea ia : Manages access and security Standard SOL 
| i O Merit Any ODBC Application— 
i FOREIGN renet w i Excel, Access and Lotus 
` Pe i i ` 5 Development Environments— 
a + Eves : Visual Basic, C/C++ 
Dica e w A Applications—Wingz, HyperCard, 
Seca” DataPrism, GQL 


To find out more about TIMESQUARE, please call us at the Soliton location nearest you. 
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